Hybrid POMDP-BDI: An Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels

Partially observable Markov decision processes (POMDPs) and the belief-desire-intention (BDI) framework have several complimentary strengths. We propose an agent architecture which combines these two powerful approaches to capitalize on their strengths. Our architecture introduces the notion of intensity of the desire for a goal’s achievement. We also define an update rule for goals’ desire levels. When to select a new goal to focus on is also defined. To verify that the proposed architecture works, experiments were run with an agent based on the architecture, in a domain where multiple goals must continually be achieved. The results show that (i) while the agent is pursuing goals, it can concurrently perform rewarding actions not directly related to its goals, (ii) the trade-off between goals and preferences can be set effectively and (iii) goals and preferences can be satisfied even while dealing with stochastic actions and perceptions. We believe that the proposed architecture furthers the theory of high-level autonomous agent reasoning

A New Approach to Probabilistic Belief Change

One way for an agent to deal with uncertainty about its beliefs is to maintain a probability distribution over the worlds it believes are possible. A belief change operation may recommend some previously believed worlds to become impossible and some previously disbelieved worlds to become possible. This work investigates how to redistribute probabilities due to worlds being added to and removed from an agent’s belief-state. Two related approaches are proposed and analyzed

Revising Incompletely Specified Convex Probabilistic Belief Bases

We propose a method for an agent to revise its incomplete probabilistic beliefs when a new piece of propositional information is observed. In this work, an agent's beliefs are represented by a set of probabilistic formulae -- a belief base. The method involves determining a representative set of 'boundary' probability distributions consistent with the current belief base, revising each of these probability distributions and then translating the revised information into a new belief base. We use a version of Lewis Imaging as the revision operation. The correctness of the approach is proved. The expressivity of the belief bases under consideration are rather restricted, but has some applications. We also discuss methods of belief base revision employing the notion of optimum entropy, and point out some of the benefits and difficulties in those methods. Both the boundary distribution method and the optimum entropy method are reasonable, yet yield different results

On Revision of Partially Specified Convex Probabilistic Belief Bases

We propose a method for an agent to revise its incomplete probabilistic beliefs when a new piece of propositional information is observed. In this work, an agent’s beliefs are represented by a set of probabilistic formulae – a belief base. The method involves de- termining a representative set of ‘boundary’ probability distributions consistent with the current belief base, revising each of these proba- bility distributions and then translating the revised information into a new belief base. We use a version of Lewis Imaging as the revision operation. The correctness of the approach is proved. An analysis of the approach is done against six rationality postulates. The expres- sivity of the belief bases under consideration are rather restricted, but has some applications. We also discuss methods of belief base revi- sion employing the notion of optimum entropy, and point out some of the benefits and difficulties in those methods. Both the boundary dis- tribution method and the optimum entropy methods are reasonable, yet yield different results

Maximizing Expected Impact in an Agent Reputation Network

Many multi-agent systems (MASs) are situated in stochastic environments. Some such systems that are based on the partially observ- able Markov decision process (POMDP) do not take the benevolence of other agents for granted. We propose a new POMDP-based framework which is general enough for the specification of a variety of stochastic MAS domains involving the impact of agents on each other’s reputa- tions. A unique feature of this framework is that actions are specified as either undirected (regular) or directed (towards a particular agent), and a new directed transition function is provided for modeling the effects of reputation in interactions. Assuming that an agent must maintain a good enough reputation to survive in the network, a planning algorithm is developed for an agent to select optimal actions in stochastic MASs. Preliminary evaluation is provided via an example specification and by determining the algorithm’s complexity

A Modal Logic for the Decision-Theoretic Projection Problem

We present a decidable logic in which queries can be posed about (i) the degree of belief in a propositional sentence after an arbitrary finite number of actions and observations and (ii) the utility of a finite sequence of actions after a number of actions and observations. Another contribution of this work is that a POMDP model specification is allowed to be partial or incomplete with no restriction on the lack of information specified for the model. The model may even contain information about non-initial beliefs. Essentially, entailment of arbitrary queries (expressible in the language) can be answered. A sound, complete and terminating decision procedure is provided

A Stochastic Belief Management Architecture for Agent Control

We propose an architecture for agent control, where the agent stores its beliefs and environ- ment models as logical sentences. Given succes- sive observations, the agent’s current state (of be- liefs) is maintained by a combination of proba- bility, POMDP and belief change theory. Two ex- isting logics are employed for knowledge repre- sentation and reasoning: the stochastic decision logic of Rens et al. (2015) and p-logic of Zhuang et al. (2017) (a restricted version of a logic de- signed by Fagin et al. (1990)). The proposed ar- chitecture assumes two streams of observations: active, which correspond to agent intentions and passive, which is received without the agent’s di- rect involvement. Stochastic uncertainty, and ig- norance due to lack of information are both dealt with in the architecture. Planning, and learning of environment models are assumed present but are not covered in this proposal

Probabilistic Belief Revision via Similarity of Worlds

Similarity among worlds plays a pivotal role in providing the semantics for different kinds of belief change. Although similarity is, in- tuitively, a context-sensitive concept, the accounts of similarity presently proposed are, by and large, context blind. We propose an account of sim- ilarity that is context sensitive, and when belief change is concerned, we take it that the evidence (epistemic input) provides the required context. We accordingly develop and examine two accounts of probabilistic belief change that are based on such evidence-sensitive similarity. The first of these switches between two extreme behaviors depending on whether or not the evidence in question is consistent with the current knowledge. The second one gracefully changes its behavior depending on the degree to which the evidence is consistent with current knowledge. Finally, we analyze these two belief change operators with respect to a select set of plausible postulates